Dynamic visualizations in multimedia learning: The influence of verbal explanations on visual attention, cognitive load and learning outcome

The main aim of this thesis was to take a closer look at visual attention allocation, cognitive load and learning outcome in learning from dynamic visualizations with accompanying verbal explanations. Instructional design guidelines derived from cognitive theories of learning with multimedia commonly recommend presenting spoken rather than written text in order to promote learning. Based on the existing evidences for the superiority of spoken over written text presentation five empirical studies were conducted to challenge the practical scope as well as the theoretical substantiation of this modality effect. In general, the studies raised two questions: (1) How do learners distribute their visual attention during learning from multimedia instruction? And (2) which design attributes moderate the effects of text modality on perception and comprehension? The studies examined several design attributes that affect perceptual and cognitive processes in multimedia learning. In order to gain direct and objective measures of perceptual and cognitive processes during acquisition, learning outcome measures and indices of cognitive load were complemented by the previously unexploited method of eye tracking. The material applied in the studies was a multimedia explanation on the formation of lightning. Besides the modality of text presentation (Chapters 2, 3, and 4) the studies varied the spatial distance between written text and visualizations (Chapter 2, Experiment 1), the visualizations being animated or static (Chapter 2, Experiment 2; Chapter 3, Experiment 1), and the pacing of instruction (Chapter 3, Experiment 2) and its control by the learner (Chapter 4). The results deliver converging evidence for an effect of text modality on cognitive load and learning outcomes under serious time constraints. However, under less attentional competition, less time constraints, and learner control of pace, these effects changed, decreased, or even disappeared. Once learners were relieved from following apparent motion or from time constrained presentation, the need to split visual attention lost much of its impact on learning. These “cognitive” effects were associated to particularities of the viewing behavior. Eye tracking measures revealed that visual attention allocation in learning from visualizations with accompanying verbal explanations follows a fairly stable pattern that was moderated by design attributes of the instruction. In general, written text dragged visual attention away from inspecting illustrations. Learners adapted to surface characteristics of the visual material (e.g. apparent motion in the visual field) and the presence and degree of time constraints by distributing their visual attention between written text and visualizations differently. Furthermore, they were able to adjust the pace of presentation to a regular reading strategy that only varies in the time taken to read text. Thus, the need to read written text may or may not interfere with extracting information from visualizations depending on how seriously reading and viewing visualizations are disturbed by the design of a multimedia instruction. As a practical consequence, the question for an instructional designer is not that much if or if not text should be presented aurally instead of visually but if the displayed information can be sufficiently extracted by an individual learner. Understanding the demands of a learning material on the learner’s perception and accounting for individual differences by implementing user interaction appears promising to advance the design of multimedia instructions in a learner-supporting fashion. Table of contents Chapter 1: Theoretical introduction and general research questions 6 Comprehending dynamic visualizations 7 The presentation function of dynamic visualizations: Does congruency help? 8 Perceptibility of dynamic visualizations: Guiding visual attention 9 Attention vs. comprehension: The role of accompanying text 10 Guidelines for the combination of text and visualization 13 Cognitive theories of learning in instructional areas 14 The role of working memory 14 Cognitive load theory 15 Cognitive theory of multimedia learning 17 Measuring cognitive and perceptual processes 19 Direct measures of cognitive load 20 Eye tracking in multimedia learning 21 Aim of the thesis: A closer look at effects of text modality 23 Chapter 2: Integrating different sources of information in multimedia learning: correspondence between viewing behavior and comprehension 26 Introduction 26 Eye movements and other process measures in settings with multiple information sources 27 Instructional design principles and theoretical explanations 29 Experiment 1: Correspondence between viewing behavior and comprehension in modality and spatial contiguity 31 Method 32 Results 34 Experiment 2: The influence of animated and static illustration on viewing behavior and the modality effect 39 Method 40 Results 42 General discussion 46 Chapter 3: Visual and cognitive load in multimedia learning: Effects of text modality, split-attention and pacing of instruction 5